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Information technology is about not only hardware, software, communication 
infrastructure and communication infrastructure but also how to manage 
services. Information technology plays an increasingly important role in 
developing the structure and functions of public and private sectors. 
Service measurement plays an important role in IT service management 
(ITSM) that is one of the subfields of Services Computing science. ITSM is a 


Keyword: big part of service science, a science field that combines computer science, 
operation research engineering, business strategy, management science, and 
Evaluation organizational theory. Performance measurement from each of IT services is 
IT-service absolutely needed and is important in the continuous development of ITSM. 
SLA These research provide good technical knowledge about the measuring ITSM 
SMS with some requirements. In this paper we suggest the metrics in each service 
Tools processes enables organizations to predict a direction for active process 
enhancement and to identify if the goal of process can achieve. This 
objective process metrics based on ISO/IEC 15504-8 and PRM ISO/TEC 
20000-4 refinement. The output of this research, in the form of metrics and 

tools for any type organizational use. 
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1. INTRODUCTION 

Information technology is about not only hardware, software, communication infrastructure and 
communication infrastructure but also how to manage services. Information technology plays an increasingly 
important role in developing the structure and functions of public and private sectors. Almost billion of 
money are being spent on the procurement of new technology with the aim to improve IT organization 
(Service delivery) as well as the quality of services delivered to customers. Although IT service often brings 
benefits to an organization, its implementation sometimes does fail [1]. 

Service measurement plays an important role in IT service management (ITSM) that is one of the 
subfields of Services Computing science. ITSM is a big part of service science, a science field that combines 
computer science, operation research engineering, business strategy, management science, and organizational 
theory. Many IT service organizations consider the measurement of IT service management processes, 
especially service processes, as a difficult task [1]. Difficulties are mainly due to the following four reasons: 
1) IT organizations do not have a structured approach for measuring IT services and service management 
processes, 2) tools used by service support teams do not enable effective measurement, 3) IT service 
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management standards and frameworks do not provide practical examples how to measure support processes, 
and 4) there are too many options what to measure in service management [2]. IT service management 
related about preparing, designing, implementing, developing, mainting IT services [2]. 

A lot of research was focusing on implementation and integration of IT service management 
processes in organizations. However, these studies mainly focus on the business and process point of view 
and technical perspective is missing. On the other hand, it is not mentioned explicitly on how to measure and 
ensure that every steps could help to increase the efficiency and effectiveness of ITSM implementation. 
Performance measurement from each of IT services is absolutely needed and is important in the continuous 
development of ITSM. These research provide good technical knowledge about the measuring ITSM with 
some requirements. This research will perform the collaboration and refinement related IT service, so that the 
matrix produced can be more practical for the implementation. 

The problem will solve in this research is about how to measure every steps in ITSM. The 
measurement is to design a framework and model integrated to each of ITSM steps, from design to 
deployment. The research problem represent in these questions: 

1. What are the best practices or framework for measuring an ITSM process with an integration and many 
progreess? 
2. How metrics can be support the best practices or framework? 


2. LITERATURE REVIEW 
2.1. IT governance 
IT governance recognized as an integral part of enterprise governance. It “consists of the leadership 
and organizational structures and processes that ensure that the organization’s IT sustains and extends the 
organization’s strategies and objectives” [3]. IT Governance is a system that aims to control and direct IT 
service activities given by the service provider. The system encompasses policy, objective, planning, process, 
documentation and resources needed in a service cycle. The cycle consists of strategy, design, transition, 
operation and continuous improvement [4]. ISO/TEC 20000-1 states that the components of service 
management consist of [4]: 
a. Strategy 
1. Management responsibility. 
2. Process governance by other parties. 
3. Documentation management. 
4. Resource management. 
5. Service management system creation. 
b. Design and transition 
1. New service design or modification. 
2. Design and development of a new or a modification of a service. 
3. Transition of a new or a modification of a service. 
c. Operation 
1. Service delivery process: capacity management, service-level management, information security 
management, service availability and continuity management, service report, service budgeting and 
cost calculation. 
2. Adjustment process: configuration management, change management, release and deployment 
management. 
3. Resolution process: problem management, incident and service request management. 
d. Continuous service improvement 


2.2. Service management system standard - ISO/IEC 20000:2011 

ISO/IEC 20000:2009 is the first international standard for IT service management (ITSM). This 
standard is based on and is intended to replace the British Standards BS 15000. This standard was first 
published in December 2005 and like its predecessor. BS 15000, was originally developed to describe the 
best practice guidance contained within the ITIL framework (Information Technology Infrastructure Library) 
standard although it also supports ITSM framework and other approaches. The latest version is currently the 
second edition of which is the development of ISO/IEC 20000:2011. ISO/TEC 20000:2011 consists of two 
main parts in the service management certification standards. The first part is a standard specification for the 
achievement of IT service management with the implementation rules for the management service. The first 
part, ISO/IEC 20000-1, advocated the use of an integrated process approach to effectively provide the 
appropriate managed service business and customer needs. The second part, ISO/IEC 20000-2, is a 'code of 
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conduct' and describes the best practices for service management within the scope of ISO/IEC 20000-1 as 


shown in Figure 1. 
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Figure 1. Service management system ISO/IEC 20000-1:2011 


2.3. Refinement PRM ISO/IEC 20000- 1:2011 

This process reference model (PRM) is a logical representation of the elements of the processes 
within service management. Using the PRM in a practical application might require additional elements 
suited to the environment and circumstances. The PRM specified in this part of ISO/IEC 20000 describes at 
an abstract level the processes including the general service management system (SMS) processes implied by 
ISO/IEC 20000-1. Each process of this PRM is described in terms of a purpose and outcomes. The PRM does 
not attempt to place the processes in any specific environment nor does it pre-determine any level of process 
capability required to fulfil the ISO/IEC 20000-1 requirements. Table 1 shows the example of refinement 
PRM for ISO/TEC 2000-1:2011. 


Table 1. Example of Refinement PRM for ISO/IEC 2000-1:2011 


Process ID SMS 2. 

Name SMS Governance of processes operated by other parties 

Purpose The purpose of the Governance of processes operated by other parties process is to 
ensure the services supplied by other parties are managed to meet the service 
requirements; 

Outcomes As a result of successful implementation of this process: 
1.The objectives and requirements for service Management are identified and 
established to satisfy business needs, the service provider's financial processes, 
regulatory, contractual and statutory. 
2.Services supplied by other parties are managed to meet the service requirements; 

Base SMS.2.1, Identify the objectives and requirements for service 

Practice management. The objectives and requirements for service management are identified 
and established to satisfy business needs, the service provider's financial processes, 
and regulatory, contractual and statutory requirements.[Outcome 1] SMS.2.2, Manage 
services provided by other suppliers. Services supplied by other parties are managed to 
meet the service requirements. [Outcome 2] 

Inputs 


12-01 Alternative party process requirements [Outcome 2] 


Outputs 


09-00 Report [Outcome 2] 
12-01 Alternative party process requirements [Outcome 1] 
09-02 Alternative parties performance evaluation report [Outcome 2] 
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2.4. Process assesment model ISO/IEC 15504-8 

Process assessment model (PAM) is a method of measuring a process consisting of indicators of 
process capability and process performance. Indicators of process capability is the ability of the process to 
achieve the level of capability that is determined by the attributes of the process. Evidence of indicators of 
process capability will support the assessment of the achievement of the process attributes. 

A PAM comprises a set of indicators of process performance and process capability. The indicators 
are used as a basis for collecting the objective evidence that enables an assessor to assign ratings. The set of 
indicators included in this part of ISO/IEC 15504 is not intended to be an all-inclusive set nor is it intended to 
be applicable in its entirety [5], [6]. Figure 2 shows the relationship between the PAM and its inputs [4]. 


Capability Dimension: 


Optimizing (2 attribute) 
Predictable(2 attribute) 
Established (2 attribute) 
Managed (2 attribute) 
Performed (1 attribute) 
Incomplete 


Process Refrence 
Model 


Process Dimension 


Figure 2. Relationship between the PAM and its inputs [4] 


Assessment activities performs to distinguish between levels 1 to a higher level. This is done 
because the level 1 determines whether a process reaches its destination, and therefore it is very important to 
achieve, and also the basis for reaching a higher level. The results obtained at each level will be categorized 
into four classifications as shown in Table 2. 


Table 1. The Four-Point Attribute Rating Scale 

Rating and Designation Description 

N (Not achieved) There is no evidence of achivement for defined attribute. The number 
: value achieved in this classification range 0-15 %. 

There is some achivement of the defined attribute. The number value 
achieved in this classification range 15- 50%. 
There is significant achivement of the defined attribute. The number 
value achieved in this category ranges from 50-85 %. 
In this classification, there is full achivement of the defined attribute. 
The number value achieved in this category range from 85-100 %. 


P (Partially achieved/ achieved in part), 
L (Largely achieved) 


F (Fully achieved / fully achieved), 


There are six capability levels [7]. At each level there is no ordering between the process attributes; 
each attribute addresses a specific aspect of the capability level [7]. The list of process attributes was showing 
in Table 3. PAM provide the basis for determining whether process attributes have achive: 

a. Capability Level 1 — Indicators are specific for each process and assess whether the following Table 3 
shows the attribute has been achieved the implemented process achieves its process purpose. 

b. Capability Levels 2 to 5 — Assessment of capability is based from generic process indicators of 
performance. These are called generic because they apply across all processes, but they are different from 
one capability level to another. 
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Table 2. Capability Levels and Process Attribute 


Process Attribute 


Capability Levels and Process Attributes 


PA 1.1 


PA 2.1 
PA 2.2 


PA 3.1 
PA 3.2 


PA 4.1 
PA 4.2 


PA 5.1 
PA 5.2 


Level 0: Incomplete process, the process was not implementing and general failur to attain its 
process purpose. 

Level 1: Performed process, the purpose of process was achieve. The achivement may not be 
tracked or planned. 

Process performance 

Level 2: Managed process, the process delivers work products of acceptable quality and 
defined by timescale. The primary of process related work products are appropriately 
established, controlled and maintained. 

Performance management 

Work Products management 

Level 3: Established process, the previously described managed process was implementing 
using a defined process that is capable of achieving its process outcomes. 

Process definition 

Process deployment 

Level 4: Predictable process, the defined process was performing consistently in practice 
within defined limits to achieve its process outcomes 

Process measurement 

Process Control 

Level 5: Optimizing process, the defined process was optimizing to meet curent and future 
businnes needs. This process also consistently in practice within defined limits to achieve its 
work products and outcomes. 

Process innovation 

Continuous optimization 


3. RESEARCH METHODOLOGY AND IMPLEMENTATION 

This research uses concurrent embedded mixed method as its methodology. Generally, this research 
uses qualitative methodology by gathering qualitative data, literatures, and elaborating some best practices or 
standards to design IT service governance. The first phase is to do literature analysis to evaluate the 
assessment and the measurement process of IT service. The analysis is a qualitative method. But, on the last 
part, a quantitative method is used to prove the qualitative method. The method is only a small part of the 
whole qualitative process. It can be inferred that quantitative method is embedded in a qualitative method. 
The said method is illustrated in Figure 3. 


Concurrent Embedded Design 


Quantitative 


Qualitative 


| 


Analysis of Findings 


Figure 1. Research mehodology 


After IT service measurement model is produced, quantitative validation is the next process on the 
line. Validation is carried on by using the model in assessing a case. Specifically, framework used in this 
research refers to a proactive research approach created by Cole R, et al. That approach has four process 
steps that needs to be done in the research. 
a. Problem identification is a process of defining a research problem and could cover the creation of initial 

concept that involves research objects. 
b. Intervention is a planning, developing, and action taking process to make a construction, model, 
prototype, or other resources. 
c. Evaluation is a process of observation and measurement of compatibility level and accuracy of produced 
output in order to support the solution of said problem. This process is a validation of model or design by 
using expert judgement. 
d. Reflection and lessons learned. This is a process for produced output. The reflection and lessons learned 
aims to report the output of the research and to find its contribution to the practice and the theory. 

In framework development phase, we start with placing the initial requirements as the direction in 

finding ITSM and IT service supporting theory literature. Theoretical model for designing metrics is showed 
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in Figure 4. The outcome of the finding will be combined by the experiences from practicioner in this field 
and will be inserted into a measurement metrics. Framework in this integration is showed in Figure 5. 
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Figure 2. Theoretical model for designing metrics 
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The next step in creating the metrics is to design Process Assessment Model (PAM) ISO/IEC 
15504-8:2013 which is based on Process Reference Model (PRM) ISO/IEC 20000-4. PRM used in ISO/IEC 
15504-8:2013 uses ISO/IEC 20000-1:2009 standard, so that researchers refines the PRM by improving its 
compatibility with ISO/IEC 20000-1:2013. It became the base of the state-of-the-art of our research. PAM 
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from the refinement is used by researcher in doing appropriate conformant assessment to the requirements of 
ISO/IEC 15504-2 and ISO/IEC 20000-1:2013. The frame of mind in creating this evaluation tools is 
illustrated in Figure 6. 

The components involved in designing a evaluation tools consist of process dimension, SMS 
processes list and capability dimension. Process dimension is a PRM process list that will be measured based 
on ISO/IEC 20000-4; each process contains the list of assessment component and the purpose to support the 
overall process of SMS. 


` Step 2 - Determine ` 
ma = + at 


Figure 6. Framework implementation 


3.1. Element assesment process 

ISO/IEC 15504-1 suggest the assessment process contains at least five specified activities: planning, 
data collection, data validation, process attribute rating, and reporting. The assessment process must be 
documented; in addition, the assessors must record the objective indicators of performance or capability used 
to justify the ratings. A team carries out the process assessment with at least one competent assessor who has 
the competencies. An assessment is carried out by assessing selected processes against the assessment 
model(s) chosen for the assessment. The assessment model(s) have to be compatible with the requirements 
defined in ISO/IEC 15504-2. The Process Reference Model is selected according to the application domain 
of interest [6]. 


3.2. Implementation tools 
A self-assessment by expert can identify process gaps that require improvements in advance of a 
formal assessment; it can be done for a relatively small investment and assists enterprise management in 
setting target capability levels. 
1) Step 1 — Deciding process to asses scoping 
Van Bon et al. (2008) explain the attributes to measure IT service process as a component. 
However, van Bon et al. (2007b) that a characteristic of modern ITSM is an end-to-end approach. The 
following attributes of an IT service can be used to measure quality: 
1. Availability, ability of a service or service component to perform its required function at an agreed instant 
or over an agreed period of time 
2. Capacity, current and forecast demand for services also expected impact of agreed requirements for 
availability, service continuity and service levels 

3. Performance, the fulfillment of a claim, promise, or request of the services at planned intervals, with the 

customer 

Security, preservation of confidentiality, integrity and accessibility of information 

Confidentiality, containing information whose unauthorized disclosure. 

Scalability, capable of being easily IT service to be expand or upgradeon demand 

Adjustability, to bring IT service or process to a more satisfactory state 

. Portability, the quality or state the IT service of being portable 
e tried to elaborate ITSM process and the attribute component into a table. 
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ITSM Process Metrics to Measure Service Process Owner 
SMS 1. Management Responsibility Availability, Chief Information Officer 
SMS 2. Governance of Processes Operated Performance, Chief Information Officer 


SMS 3. Documentation Management 


Adjustability, security, confidentiality 


Release Manager 


SMS 4. Resource Management Capacity, performance, availability Capacity Manager 
SMS 5. Establish the SMS Availability, capacity, adjustabilit Service Desk Manager 
DTR 1. General Performance, portability Capacity Manager 


DTR 2. Plan New or Changed Services 
DTR 3. Design and Development of New or 
Changed 


DTR 4. Transition of New or Changed Services 


SDE 1. Service level management 
SDE 2. Service Reporting 


SDE 3. Service Continuity and Avaibility 
Management 

SDE 4. Budgeting and Accounting for Services 
SDE 5. Capacity Management 

SDE 6. Information Security Management 
REL 1. Business Relationship Management 
REL 2. Supplier Management 

RES.1 Incident and Service Request 
Management 

RES.2 Problem Management 


Capacity, performance, availability 
Capacity, performance, 
Availability, scalability. 


Capacity, performance, availability, 
adjustability. 

Availability, capacity 
Availability,capacity, performance, 
security 

Availability, performance 


Capacity, adjustability 

Capacity, performance 

Security, confidentiality, avaibility 
Avaibility, performance 
Avaibility, performance 

Security, confidentiality, avaibility 


Security, confidentiality, avaibility, 
capacity 


Configuration Manager 
Configuration Manager 


Configuration Manager and 
Change Manager 
Release Manager 
Release Manager 


Release Manager 


Release Manager 
Release Manager 
Release Manager /CSO 
Relationship Manager 
Relationship Manager 
Incident Manager 


Problem Manager 


CON.1 Configuration Management Capacity, confidentiality, avaibility Change Manager 
CON.2 Change Management Scalability, portability Change Manager 
CON.3 Release and Deployement Management Scalability, portability, adjustability Change Manager 


The first step in the self-assessment is to decide what processes will to assess. Those processes 
selected and to have record in the table as shown in Table 5. 


Table 4. Capability Levels and Process Attribute 


Process Name Level 1 Level 2 Level 3 Level 4 Level 5 
PA 1.1 PA 2.1 PA2.2 PA3.1 PA3.2 PA4.1 PA4.2 PA5.1 PA5.2 
Target F 
Rating by Criteria 
Rating 


Capability Level Achieved 


At this stage, the target process capability level can be recorded. This will establish the level of 
capability required of the process. In setting the target capability levels, consideration will give to the impact 
on the business objectives of the enterprise if a specified level of capability will not achieve. The first 
consideration is the impact on the enterprise if the process is non-existent or not working effectively or 
efficiently. 

2) Step 2 - Determine Level 1 Capability 

The first step in the assessment of each process is to determine whether a process is actually being 
performed and is achieving its outcomes. In the self-assessment worksheet Table 4 there is a table for each 
process. The indicators at capability level 1 are specific for each process and assess whether the following 
attribute has been achieved: The implemented process achieves its purpose. 

3) Step 3 - Determine Capability for Levels 2 to 5 

Above level 2, the assessment criteria are generic, i.e., the criteria are the same for each and 
every process. Again, in each case, a judgment must be made as to whether the criteria have been met, and 
that decision must be translated into a rating and recorded in the template for the process. This should be 
repeated for each capability until a capability level is rated as ‘largely’ or ‘fully achieved’. 

4) Step 4 - Record and Summarise the Capability Levels 

The summary of assessment results should be recorded in Section 1. The capability level is 
determined at the level where both capability indicators are either ‘largely’ or fully achieved’. Those 
processes selected should be recorded in the table as shown in Table 4. 
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5) Step 5 - Plan Process Improvement and Make Recomendation 

Based on the self-assessment, consideration should be given to the development of a plan of action 
for process improvement. One option could be to commence an initial mprovement plan based on the self- 
assessment. This could address the areas of highest importance to the enterprise’s business goals and focus 
on areas with gaps between the ‘current’ and ‘target’ process capability levels. 


3.3. Implementation 

We tried to the implement measurement metrics in an organizational unit of ICT service 
management in a higher education institution. Scope definition of evaluation by using this metrics 
encompasses main business processes of ICT services provided by the organization. The organization, 
currently, is functioning as a system support in delivering the services related to those activities. 

The ITSM proceses - capability level assessment is carried on by gradually evaluate whether the 
processes are fulfilling the requirements on every level, from level 1 to level 5. There are category provisoins 
of assessment result on every level, those are largely achieved (L) with range of score between 50-85%, and 
fully achieved (F) with score range between 85%-100%, so that the process can be declared that it has 
attained a capability level. The assessment is performed until the process achieve the expected level set by 
the organization. 

From the outputs of metrics as shown in Figure 7. We argue the true value of IT service provision 
can only be realised when the services it provides are solutions to identified business needs that are both 
practical and reliable. In order to achieve this, these services need to be well managed. This improvement in 
the provision and management of services promotes the credibility of the industry while improving customer 
loyalty and satisfaction. Service providers must ensure their capacity to provide and manage identified 
services. This introspection is best performed in an environment that does not contain a customer. 
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Figure 3. The assesment result using proposed metrics 


4. CONCLUSION 

In this paper we suggest the metrics in each service processes enables organizations to predict a 
direction for active process enhancement and to identify if the goal of process can achieve. This objective 
process metrics based on ISO/IEC 15504-8 and PRM ISO/IEC 20000-4 refinement, which has not been 
introduced in previous process assessment models, can be expected to measure process capability and to 
identify the risk, problems, and condition of process performance by using these metrics. This metrics 
provides a basis for use in IT service process improvement. From the results of this paper, following 
advantage can be obtained: facilitates self-assessment, produces a process rating for every ITSM process, and 
can raise formation's IT service management system process capability level. This metrics has already testing 
by author with implementation and justification by ITSM Expert at Polinela (higher education institution). 
The output of this research, in the form of metrics and tools for organizational use, can be downloaded in 
http://bit.lyITSMtools2018. 
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